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1. Introduction 



Several years ago some authors in the US stated that there is a 
problem with the economic literacy of the American people. E.g. George 
J. Dawson reports in 1975 from several studies that even simple 
economic questions could not be answered by many people (including 
high school teachers of social studies!). We can't tell the same for 
the German speaking countries. The reason is not that people in these 
countries possess a high standard of economic literacy but rather that 
we don't know anything about the state of economic knowledge of our 
populations. 

We have the strong feeling that this situation should not be 
prolonged. Like the US we are faced with a lot of problems, e.g. 
pollution, unemployment, social difficulties which mostly have an 
economic impact and to which every individual contributes by hiw or 
her actions if only to a marginal extent. We hope that one of the 
outcomes of a higher standard of economic literacy is that more people 
are able to judge their own actions as a little but nevertheless a 
relevant contribution to the aggregated effects which they don't want 
to be confronted with. 

We decided to follow the precedent set by our colleagues in the US: to 
measure first the extent of economic illiteracy in the German speaking 
countries. In the long run it could be of interest to be able to 
compare the outcomes of such an investigation between our countries 
not only because of academic curiosity but also to have informations 
about the effects of economic literacy on macro-economic outcomes, as 
well as the effects on the welfare of every individual. 

Therefore we were glad to discover that in 1987 the 2^^ edition of the 
Test of Economic Literacy (TEL) by Soper and Walstad was published. We 
are very grateful to Bob Highssith who not only sent us a copy of this 
instrument but also encouraged us to do this work and also to Bob 
Walstad who provided us with important and interesting material. 
Though we have completed to date only the first steps, we are pleased 
to be able to report our findings and to discuss some problems. 

What we have done and what we can describe is, first, the translation 
of the TEL, second, the analysis of data we collected to judge the 
quality of our German version which we call "Wirtschaf tliche Bildung 
Test" (WBT) and, third, a short discussion of some results. These 
results are still rather uncertain but they allow a first glance at 
dimensions and directions of the information we expect to obtain when 
the comparison is completed « 

A lot of things remain to be done and we regret that we can only 
present some initial indications of the level of economic literacy in 
our countries compared wi^;h the American sample for the norming of 
TEL. Also, we are not able to report sufficient data to all^pw a 
reliable comparison between Austria, Switzerland, and West Germany. 
One of the reasons for that is that our very first set of data showed 
some stramge features, e.g. negative discrimination scores which we 
couldn't interpret. So we had to reanalyze everything done to date - a 
procedure which was time consuming especially because of the distance 
between our living and working places which is some 400 miles. In 
between we have, as you'll see, better results with an expanded sample 
which is nevertheless too small to answer all questions of interest. 
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2. Problems of translation 
2.1. The strategy 

There are 77 different items on both forms of the TEL, 15 of them 
being anchor items which are contained in both forms A and B. As a 
principle of adaptation we decided to translate, as far as possible, 
word by word and sentence by sentence, and only to deviate from this 
strategy if unavoidable. The reason for this was that we intend to get 
comparable data and we would miss this aim if we had items with rather 
different sentence structures. As you know, the level of cognitive 
achievement necessary for solving a problem is, among others, a 
function of the complexity of the text structure in which it is 
presented. 

On the other hand we had to change some typical constructions normally 
used in the American language but not familiar in our language. E.g. 
in German the -ing-fom cf verbs must sometimes be expressed with a 
whole sentence which makes the text structure objectively a little bit 
more complicated but is at the same time quite normal for German 
speaking people and seems to be compensated for by a gain in 
redundancy (cf. A 16 A, B 39). In this case we hope that the 
translation doesn't affect the subjective difficulty of the text. 

Another problem is that we cannot always divide sentences in the same 
manner in one pre-part and ^everal post-parts to permit multiple 
choices for a question. As in the American language sentence 
construction is following the SPO-rule, in German we are using the 
SOUrrule. So, if in an item the "O's" content the choice possibilities 
you can express the item text in English in good style but in German 
the same construction sounds more or less strange and would therefore 
need some more cognitive processes to be reconstructed in an 
understandable way. In cases like that we sometimes had to rearrange 
the answers iuto a form which is not equal but, as we hope, equivalent 
to the original TEL-text (cf . A 4, A 13, B 20, B 29) . 

In following this strategy, which we describe short by the expression 
"as literally as possible, as free as necessary" we produced two 
translations for both forms of the test, one in Nuernberg, West- 
Germany, and one in Salzburg, Austria. * Both groups worked independent 
from each other. Then the results were exchanged and compared. It 
turned out that the differences were not very great regarding the 
sentence structure problems. However we had to struggle with greater 
difficulties in the transfer of some economic constructs which means 
the transfer of the scientific terminology. More about that is 
reported in the next sections. The differences in the grammatical 
construction of sentences were discussed between both groups and then 
resolved by agreement on one of the proposals. In some cases we found 
by the discussion a third solution. 



2 This work was done with the help from Patricia Rieche, 
Nuernberg, Lolli Gurtner, Romana Schiller, Elfi Wessenmeyer, 
Q Salzburg. 
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2.2. Lexicographic problems 



As just mentioned^ one main point of concern was the translation of 
the economic terms. The sense of the test items and the correctness of 
the solutions depend strongly on a very careful handling of this 
problem. Though there are some dictionaries in the field of economics, 
we also asked some colleagues in the economic department of the 
University of Nuernberg who are specialists in comparative economics 
to check the WBT with respect to this point. ^ 

There were, for example, some difficulties with the Meaning of 
"residential property tax" (A 24 D) for which we have no defined term 
in the Gernan science of economics. The same is true for "margin 
requirements" (A 35 B) . In both cases we kept the sense of the 
original text in the proc*»ss of translation. But we must remember the 
fact that our people are not used to reflecting on these concepts as 
Americans does. Iiooking at the distributions of percentage responses 
on these two items you can see that the two pairs of patterns show a 
remarkable difference. 

Item TEL WBT 



Tab. 2-1: Percentages of response to items A 24 and A 35 

We cannot exclude the possibility that the reason stems partly from 
the fact just mentioned. 

Another dimension of the language problem becomes visible if one looks 
at those concepts which are used as foreign words in the German 
scientific terminology stemming from the American language. For 
example we use in German the term "Opportunitatskosten" for the word 
"opportunity cost" (cf . A 2) and "komparativer Vorteil" for 
"comparative advantage" (cf. A 40). So, for the less educated German 
speaking people these words are strange because they have no meaning 
in colloquial speech. If you are one of those who have a little but 
not an extensive knowledge about economics and if you are an American 
native speaker you might have a chance to finu the correct answer for 
the item concerned on the basis of your common sense understanding of 
such a term. But there is no equivalent opportunity for the non- 
American speaker. 

On the other hand it makes no sense in these cases to try to find a 
translation based on colloquial German speech because in this case the 
well-educated person with good knowledge in economics would never 
understand what is meant by such a translation. So we decided to keep 
the foreign language term but also to remember this fact when we 
collect data on the difficulty of these items. 



A 24 
A 35 



A B C D 
14 12 60* 13 
21 24 33* 20 



bl 
1 

2 



A B C D 
28 14 49* 6 
16 14 43* 18 



bl 

3 
9 



We thank Dr. K.-D. Grueske, N. Leineweber, and W. Heinzmann for 
their help. 



Actually we found only slight differences in the response patterns of 
the TEL and the WBT; 



Item 



TEL 



WBT 

A B C D 

16 15 15 49* 

10 17 53* 12 



A 2 
A 40 



A B C D 
7 15 24 53* 
15 22 50* 12 



bl 
1 

2 



bl 
5 
8 



Tab. 2-2: Percentages of response to item A2 and A40 

But looking at the differences between the American group "without 
economics" and the German speaking group with a lower level of 
economics education and also a lower level of time spent at school one 
can see that students might have had a problem with foreign words. The 
discrimination scores for both groups are, respectively: 

Item TEL WBT 

A 2 36.1 15.9 

A 40 41.9 36.4 

Tab. 2-3: Discrimination scores for item A 2 and A 40 
2.3. Problems with intercultural differences 

No doubt the differences between our countries with respect to their 
cultures are not very great. In Europe we are used to say that it 
takes about five years until the new American ideas are imported and 
imitated by the greater parts of our population. But there remain many 
uniquenesses e.g. in school systems or legislation or ethnic 
population patterns etc. So we had to deal with the problem of 
cultural differences in this wide sense of the word. For example Sandy 
Smith who is "Sabine Schmidt" in German (cf. Item A 5) in our 
countries does not have the choice of taking a job payini $ 10.000.- a 
year (we usually compare salaries on a monthly basis) and going to a 
college which costs $ 5.000.- a year (schools in onr countries are at 
this level normally free). Nevertheless, we kept the sense of this 
item too and adapted it only to our education system speaking of an 
apprenticeship as a hair stylist and of a private school for economic 
education which is not often but yet sometimes found in our countries. 

Another example: The labor unions in our countries play a rather 
different role in economic conflicts and are organized in another way 
than they are in the US. So "effective labor unions" which is a 
distractor in Item A 7 has in German another meaning and therefore our 
people think of other facts when they read this text. 



Which of the following is the most essential for a market economy? 

A. Effective labor unions. ^ 

B. Good government regulation. 

C. Responsible action by business leaders. 

D. Active competition in the marketplace. 

Tab. 2-4: Item A 7 in the TEL 



Item A 7 
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In this case we could keep the formulation because the different facts 
the readers are thinking of are both incorrect with respect to the 
question put. 
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Similar but more serious problems with the term "labor unions" arise 
within item A 11 where the special understanding of function of US- 
labor unions is asked for. 

Item A 11 

Labor unions in the United States have 

A. improved the bargaining position of unionized workers with 
their employers. 

B. greatly increased wages of union workers compared to nonunion 
workers. 

C. organized a majority of the American labor force, 

D. increased competition in the labor market. 

Tab. 2-5: Item A 11 in the TEL 

Here we had to decide whether to keep the literal sense of the 
question and ask German speaking people something about the conditions 
in the US, or whether to substitute this item with an analogous one 
asking for the functions of labor unions in our countries. Our 
solution for this item was to delete the reference to the United 
States in the pre-part of the sentence and to ask for the functions of 
labor unions in general. We found a reformulation for distractor C 
which now states that the working time of union workers was shortened 
compared to the working time of nonunion workers by the labor unions, 

Ifs not possible to discuss all such problems here, but two further 
examples may be mentioned. In Item A 15 and A 21 the question text 
resp, the answer refers to the "minimum wages law" which we don't have 
in our countries. And in Item A/B 17 the TEL speaks about baseball, a 
sport which is not very popular with us and in which you can't get a 
lot of money. In this last case we decided to substitute baseball with 
tennis which became very popular in Europe in the last years. But we 
are nevertheless not sure whether our kids and young people thinking 
of tennis have the same connotations as their peers in the US and if 
not what that means for the individual chance to find the correct 
answer. 

Problems of this "cultural type" arise in the following items: A 5, 
11, 15, 17, 20, 30, 31, 43, 44; B 2, 17, 18, 31, 43, 44, 

2.4, Problems with cognitive reference areas 

There are some items on the TEL which speak about the US or make 
references on other countries, E,g, item A 1: "When the United States 
trades wheat to Saudi Arabia in exchange for oil or in Item A 44 
which focuses on appreciation of different currencies. As already 
mentioned above, the problem with these items is that we have to. take 
into account that the connotations which arise in the minds of people 
tested are somewhat different and therefore influence their answers. 
This does not only mean that there may be differences in the cognitive 
dimension which e,g, are based on the fact that it could be easier to 
know "everything" about such a little country as Austria, Switzerland, 
or West-Germany, whereas the US-citizen has to think about more facts 
and relations when he or she is looking at the very complex situation 
which is connected with the name "United States", 
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The other point is that we have to reflect on the fact that the 
attitudes associated with the name of one's mother country (we say in 
German "father country"!) may be very different between our 
populations and may interfere with the process of actualizing the 
relevint knowledge. E.g. we can say that the average citizen of 
Switzerland is very proud to be a member of his society, whereas the 
things are more difficult in West-Germany and Austria where often the 
opposite may be true, and in the US where you may find more and harder 
contrasts in the attitudes associated with the name of the mother 
land. Beeing asked, what's the best to do for your own country under 
an economic aspect, those attitudes may interfere with the 
reproduction of economic knowledge. Again, we don't know what the 
effects of these differences are, but we feel that they should be 
thought about in interpreting the different results of administering 
the TEL and the WBT. 

2.5. Special problems 

In the process of translating we discovered some difficulties in the 
formulation of items. We are not sure whether these difficulties arise 
only in the German version or whether the are already hidden in the 
original TEL. E.g. look at Item A 2: 

The opportunity cost of a new public high school is the 

A. money cost of hiring teachers for the new school. 

B. cost of constructing the new school at a later date. 

C. change in the annual tax rate to pay for the new school. 

D. other goods and services that must be given up to build the 
new school. 

Tab. 2-6: item A 2 in the TEL 

The first part of this statement focuses the attention on the meaning 
of the term "opportunity cost" which is defining a special kind of 
costs. Consequently, answer A and B offer different sources of costs 
corresponding with the example uoed. But C and D don't contain ^.he 
term cost. C is speaking about some "change" which might be a hidden 
signal for the wrongness of this distractor reducing its chance for 
being selected. And D, the correct answer, doesn't use the term cost 
either - a fact which, at least in the WBT, leads to a formulation 
which doesn't sound correct because "goods and services" are not the 
same as "costs". Besides that the grammatical construction of D is not 
a correct completion of the initial phrase. 

Or think of Item B 40: 

Which of the following best describes what the law of comparative 
advantage means for the trading nations? Each trading nation . can 
benefit by selling those goods 
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A. that it produces at high opportunity costs and buying those 
goods it would produce at low opportunity costs. 

B. that it produces at low opportunity costs and buying those 
goods it would produce at high opportunity costs. 

C. that people enjoy least, while buying goods that they enjoy 
most. 

D. at comparative prices and buying others at comparative 
prices. 

Tab. 2-7: Item B 40 in the TEL 

Here the correct answer is B. It shows the same grammatical structure 
as A and C in completing the preceding part of the sentence. We felt 
that the distractor D is not very good because there is no special 
reference term for "those goods". At least, in the WBT the answer D 
doesn't match very well with the first part of the whole answer. 
Another problem with this item lies in the distractor C. In our 
opinion it is not clear enough whether the people mentioned here are 
the people of a special trading nation or all people in general, again 
with the consequence that for this distractor thfc probability to be 
chosen goes down. Again, we were therefore not very surprised that the 
percentage of responses for these two distractors is rather low: 19% 
and 16% in the TEL, 16% and 22% in the WBT. 

A third and last point should be mentioned here. It's Item B 46 which 
caused some difficulties for the translation, more precisely, answer 



Item B 46 

Rapidly growing economies usually have a 

A. slow rate of inflation. 

B. high rate of investaent. 

C. low rate cf population growth. 

D. high rate of population growth. 

Tab. 2-8: Item B 46 in the TEL 

In German economic terminology we can speak of a "low rate of 
inflation" but not of a "slow rate of inflation". We confess that for 
a short moment we were not sure whether this formulation was only a 
mistake in the TEL because with "low" as the first word of distractor 
A the answers A to D would show the pattern "low"-"high"-"low"-"high" 
at their beginnings. But then we learned that you also can - shortly - 
speak of "slow rates". So for a better comparison we kept the word 
"slow" in the WBT, hoping that the tested persons would produce the 
appropriate cognitive associations. Looking at our results, we are now 
not quite certain about the effect of this problem: 

Item B 46 

« 

A B C D bl 

TEL 18 40* 12 26 4 

WBT 15 60* 7 13 5 

Tab. 2-9: Percentages of response to item B 46 
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For this and the other problems mentioned above we'll have to do 
special investigations to get better knowledge about th« height and 
the direction of the influence on results caused by translation. 



3- The Foraal Similarity between the TEL and the WBT 

As was mentioned in Section 1, we were concerned in the first step of 
the study ^ith the translation of the TEL so that the WBT would be as 
similar as possible. The WBT is supposed to gather the saute kind of 
information in German-speaking countries as the TEL is supposed to 
validly measure in the USA, namely, economic literacy. 

Which characteristics can lead one to conclude that these goals have 
been reached? We looked for criteria for similarity or comparability 
on two levels: 

1. on the level of the instrument per se 

2. on the level of that which the TEL and the WBT measure 



A B 





TEL 


WBT 


TEL 


WBT 


Cronbach's Alpha 


.87 


.86 


.88 


.88 


Std. Dev. of mean 


8.45 


7.87 


8.85 


8.28 


Std. error of meas. 


3.06 


2.87 


3.04 


2.80 


average Item-Totai- 










Correlation (Median) 


.32 


.38 


.34 


.44 


No of higher Discr. 


16 


30 


12 


34 


Number of Dimensions 










in Factor Analyses 


1 


1 


1 


1 



Tab. 3-1: Comparative aggregate statistics for the TEL 

(Soper/Walstad 1987, 12) and the WBT (only students with 
courses in economics)^ 



From a (formal) perspective the two instruments are adequately 
similar. However, this unfortunately only indicates that they only 
fulfill the necessary requirements for any given instrument of 
measurement. This similarity is, of course, not sufficient. For that 
reason ;fe would like to now discuss the aspect of validity. 




This w( k was done with the help from H. Astleitner, Dr. G. 
Haider, Salzburg and Chr. Sczesny, A. Mann, Nuernberg. 
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4. The content similarity of the TEL and the WBT 
4.1. Econonic literacy in the TEL 

V.\n TEL is based on the following content-cognitive matrix: 





Cognitive Level 


No. of 
Quet- 
llont 


Per- 
cent 




1 


n 


111 


IV 


V 


FundofTMrtol Cco.^Mnlc Concvpit 












12 


26 1 


1 Scarcity 




25. 3 








2 




2. Opoortunity cost/hodfi-orrs 






2.4 


& 




3 




3 Producnvtty 




6 






8 


2 




4 Economic «yst«mt 


7 










1 




5. Economic Immutlcns & InctntlvM 


9 


1 1 








2 




6 Exchong«.mon«y.&r(t«rd«p«ndonco 


12 




I 






2 




Mlcro«conoml9 Economic Conc«pte 












13 


28 3 


7. Martt«ttatp(lCM 




10 


13 






2 




8. Supply A d*mond 


14 




15. 16 




17 


4 




9 Compolttlon k mofk«t structur* 






18, 19 






2 




10. lncom« ditribunon 


20 


21 








2 




n Mart(«tfblurM 








22 


23 


2 




A7. Rol« of gov«rnm«ot 




24 








1 




Macfo«conomle Economic Concopis 












13 


28 3 


13 GfOtt notional product 


26 










1 




U. Aggrogoto m43ply 




27 








I 




15. Aggcogoto domond 




28 




29 




2 




16 Unomploymont 








30 








17. Inflation & doflotlon 








31.32 




2 




16 Monofofypdcy 




34 


35 




38 


3 




19 Rscd poicy 


36 






33. 37 




3 




tnt«motlonal Economic Conetpti 












8 


174 


20. Compofatlvo o<Mintog9/bomorj to trodo 




39 


40 


41 




3 




21. Bolonco of payments & •xcnongo rates 


42 






43.44 




3 




22. International growth & stobtity 




45.46 








? 




Totol No. OfOuetttons 


8 


13 


10 


11 


4 


46 




Percent of Total 


\7A 


28 3 


217 


23 9 


87 




100 



Tab. 4-1: Content-Cognitive Matrix: TEL Form A 

The cognitive levels are defined by Bloom (1979) and are not of 
interest here. The content concepts are from the Master Curriculufn 
Guide: A Framework for Teaching Economics: Basic Concepts. (Source: 
TEL 1987, 3) . 



It is therefore not surprising that the content validity appears to 
have been rated positively by the ten-member National Advisory 
Committee of prominent economists. » "Economic literacy" is therefore 




Unfortunately neither qualitative nor quantitative data were 
found on the experts* ratings or their suggestions for 
improvement. 

n 
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understood by test authors as well as by the Advisory Committee to be 
that which the TEL measures.* 

4.2. Economic literacy in the WBT 

The approach taken by the TEL authors suggested that the content 
validity of the WET also be assessed by experts. We asked the 
following question to experts at six German, one Swiss and one 
Austrian university: 

"Please judge. ..each test item on how well or how poorly it seems to 
you to be an indicator of economic literacy (among 18 year-olds)." 

We would need to write an entire paper to present the results in 
detail. Here, we will have to limit ourselves to the following 
findings: 



6 We have not been able to find in any of the literature on the 
TEL a reflection on how the concept 'economic literacy' is 
defined or on which alternatives could be considered. Scheer is 
the only author who indirectly defines economic literacy: 
"Economic illiteracy is defined as a lack of understanding of 
how the American economic system functions and a deficient 
degree of knowledge concerning basic economic concepts and 
relationships." (L.H. Scheer: Economic Literacy: Goal of 
Centers for Economic Education. In: Indiana Social Studies 
^ Quarterly, Aug. 1974, 27/2, 66-73. Note 1, Page 73). 
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Form A Form B 



Item 


Mean 


Std Dev 






1 


3.5 


2.2 


3.4 


9 0 


2 


4 3 


1.8 


4 0 


9 n 


3 


2.5 


1.4 


4 7 


9 1 


4 


3.1 


1 2 


J . y 


0 1 


5 


3.1 


1 6 


J . o 


L . U 


6 


3.5 


1 7 


1 1 
J . J 


O A 

Z • u 


7 




1.7 


J . J 




8 


4.4 


2.1 




1 d 


9 


2.4 


,7 


2.9 


1 8 

X . o 


10 


3.2 


1.8 


3.0 




11 


3.6 


2.0 


3.1 


1 3 

X . «/ 


12 


4.0 


1.7 


9 9 


1 1 


13 


1.9 


1.1 


9 *> 
It % It 


1 R 


14 


2.6 


1 1 


'K 1 
J . I 


0 9 
Z . / 


15 


3.1 


1 8 

X . u 


^ . 3 


1 n 


16 


3.2 


1 6 

X . v 


\ 0 

J . u 


1 ^ 


17 


3.8 


2.4 


3 6 


1 Q 


18 


3.1 


1.8 


2 9 

£t • J 


1 4 


19 


2.9 


1.7 


2.1 


1.7 


20 


4.6 


1.7 


3.1 


1 8 

X . o 


21 


3.5 


1.8 


2.9 


1 3 

X . «/ 


22 


4.0 


2.0 


2 7 




23 


3.5 


2.0 


3.1 




24 


2.8 


1.2 


2 9 


1 Q 


25 


3.4 


1.8 


3.3 


1 9 

X . 7 


26 


J.O 


2.3 


2.3 


1 4 

X . ^ 


27 


3.6 


1.6 


3.3 


1 . 5w 


28 


4.0 


1.8 


3.0 


1.2 


29 


4.0 


1.6 


3.2 


1 5 


30 


2.4 


1.5 


2.4 


1 3 

X . 


31 


2.4 


1.6 


it • J 


1 9 


32 


3.1 


1.9 


3.4 


1 7 


33 


3.1 


1.6 


2.6 


1 5 

X . «i 


34 


3.0 


1.6 


3 0 




35 


3.3 


1.6 


3.4 


1 6 

X . u 


36 


3.2 


1.5 


2 8 
it • %i 


1 9 

X, • It 


37 


2.6 


1.0 


2 8 


X.J 


38 


3.4 


1.7 




X.J 


39 


4.0 


1.7 




1 7 

X . f 


40 


4.1 


1.8 




1 1 

X.J 


41 


3.5 


1.5 


3 0 


X . t 


42 


3.1 


1.1 


2.7 


1.7 


43 


3.6 


1.4 


3.8 


2.0 


44 


3.1 


1.8 


3.4 


2.0 


45 


3.1 


2.0 


2.8 


1.6 


46 


4.0 


2.0 


3.8 


2.1 


Average 


3.15 


.70 


3.3 


.77 



Tab. 4-2: Ratings of German-speaking experts regarding the 
question: "How well do the WRt^ nd the TEL 
respectively) measure econoD iteracy?" (1= very good, 
7= very bad, 4 = 'mean') 
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We interpret the data in the folloving manner: 

1. The fact that the total averages with a rating of 3.15 (SD .70) for 
Form A and 3.3 (SD .77) for Form B lie on the left half of the scale 
is a positive aspect of the results. On the average the experts agree 
that the WBT (and therefore the TEL) does measure economic literacy. 

2. One unpleasant result, however, are the wide variances on many 
items (Table 4-2). In some cases they are so great that an arithmetic 
mean is questionable. That leads one to suspect that there is no 
unified concept of economic literacy for 18 year-olds among the 31 
polled experts. 

3. This lack of unification is expressed in another unpleasant 
finding: only 11 of the 31 German-language university experts are of 
the opinion that, with the 46 test items, all dimensions of economic 
literacy which are expected for 18 year-olds are addressed; 20 - more 
than half - are of the opinion that essential aspects are missing. 
Without going into more detail, we list here briefly which aspects are 
missing in the opinion of those who were dissatisfied. The key words 
give you a rough idea about what German-language experts also think 
about when they reflect on the concept of economic literacy. 

1. Too one-sided (neo-classical) macro-economic orientation 

There are questions lacking on 

+ other than market economic systems 

+ economic policy/development policy 

+ social market economy (soziale Marktwirtschaf t) 

+ economic structural data / budget / GK? 

+ foreign economy / international economic organizations 

+ economic history / technological progress 

+ economic ethics 

+ limits of market economy 

2. Too little micro-economic orientation 

There are questions lacking on 

+ production, sales / marketing 
+ financing / stock exchange 
+ types of companies 

+ daily problems such as: financial transactions, income 

tax, social security, rent, purchasing contracts ... 
+ finaclal ratios (productivity, profitableness, liquidity) 
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3. Other criticisms 



+ the items are too difficult for 18 year-olds 

+ questions are obsolete 

+ distractors are occasionally not clear 

+ irritating repetition of questions 

+ mere questioning of definitions 

+ no concept of economic literacy recognizable 

Tab. 4-3: A brief qualitative overview of the 'essential' 

weaknesses of the WBT as an instrument for measuring 
economic literacy,'' 

What does this content criticism make allusion to in as much as it is 
justified and the experts have not simply overlooked that some of the 
demands have indeed been met? Does it allude to the question of which 
kind of economic literacy 18 year-olds shoald have or to the question 
of whether 18 year-olds can lave the kind of economic literacy 
implicitly defined by the TEL or WBT, 



As the overview shows, statements can be made concerning both 
questions. Criticism of the inherent normative concept of economic 
literacy in the TEL clearly dominates and with that, the Master 
Curriculum Guide is indirectly criticized. This leads to the 
interesting question of whether American and German-speaking 
university experts understand the concept of economic literacy 
differently; however, this is not the place to discuss this question. 
The only significant question is whether the concept of economic 
literacy as defined in the WBT is, in principle, appropriate for 18 
year-olds. 

In view of this question we will rely for the time being on the polled 
experts' relatively good quantitative ratings mentioned previously 
(compare Table 4-2) . 

To repeat, on the average the experts indicate that economic literacy 
is sti31 acceptably defined by the TEL and the WBT. However, the test 
has gaps in its content but can, as a whole, still be considered 
adequately content valid. 

Nevertheless, this rating is not sufficient for the purpose to which 
we aspired: comparing American and German-speaking young people with 
respect to their economic literacy. If American and German experts 
rate the content validity of the TEL and the WBT positively, it is not 
necessary for the two versions to resemble each other. It is 
theoretically conceivable that the ratings apply to different implicit 
concepts of economic literacy (Figure 1): 



We are not able to show here how the good and poor ratings 
stand in relationship to the criteria of the experts. We will 
report on that issue elsewhere. 



15 
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Fig. 4-1: Theoretical (Possible) relationship between the American 
and the German concept of economic literacy 

According to this model American experts rate all the items but 
German-speaking experts only the items in the intersection positively. 
Several items are not included in the German-speaking experts' concept 
of economic literacy. For that reason tliey rate them negatively. At 
the same time, they find other dimensions missing which are referred 
to in Table 4-3. 

This is speculation of course; however, the possibility of it being 
valid can not be ruled out. Th'^refore, our study had to concentrate on 
determining if the TEL and the WBT measured in the same way whatever 
aspect of economic literacy they waited to measure. The question is 
then whether the WBT and the TEL measure comparable things. 

4.3. Similarity of reactions of Aaerican and German-speaking young 
people to the TEL and the WBT 

We assumed that the similarity of reactions of American and German- 
speaking students while working on the TEL or the WBT would give 
information about whether the TEL and the WBT measured comparable 
things. Our questions were as follows: 

+ Are items which are difficult for American students also 
difficult for German-speaking students? 

+ Do the items in the TEL and the WBT tend to be answered 
correctly by strong students and incorrectly by weak 
students? 

+ Do the German-speaking students react to the distractors in 
the same way as the American students? 



ERIC 
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1. Correlations of Dif f iculuties in TEL und WBT 

TEL-A TEL-B 
WBT-A .62 

P= 0.000 

WBT-B .62 

P=o.ooo 

2. Correlations of Discrimination in TEL and WBT 

TEL-A TEL-B 

WBT-A .35 

p=.ol8 

WBT-B .40 

P=.oo5 

Tab. 4-4: Similarity of reactions of American and German-speaking 
students while working on the TEL or the WBT 
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Item 


Form A 


r ur ui 


1 


1.00 


. o\j 


2 


80 


on 


3 


1.00 


. 0 J 


4 


.95 




5 


.80 


. OK) 


6 


.40 


. 


7 




1 f\f\ 


8 




• oU 


9 


1 00 




10 


.80 




11 


.40 


95 


12 


.20 


. 0 J 


13 


.80 


. o\/ 


14 


.80 




15 


.63 


1 .00 


16 


1.00 


95 


17 


1.00 


1 00 


18 


.77 


1 00 


19 


• 80 




20 


.63 




21 


40 


90 


22 


.40 


1 00 
1 . \)\) 


23 


.80 


ftO 


24 


.80 


1 00 


25 


-1.00 


80 


26 


.63 




27 


1.00 


95 


28 


,63 


. 


29 


.32 


PO 


30 


95 


fin 

• OU 


31 


95 


fiO 
. OU 


32 


- 60 


. flU 


33 


80 




34 


.63 


40 

. 4U 


35 


.20 


1 00 


36 


.95 


32 


37 


1.00 


32 


38 


.80 


f)0 

. OU 


39 


40 


1 00 

1 . UU 


40 


80 

. V V 


90 
. aU 


41 


.40 


40 

. TV 


42 


.80 


1 00 

X . vv 


43 


.74 


.95 


44 


1.00 


1.00 


44 


.80 


.63 


46 


.32 


.80 



Average: 0.81 0.90 

Tab. 4-5: Average correlations of answer distributions in the TEL 
and the WBT by item (Kendall's tau) 



The tables show that, without a doubt, there is (even before the items 
were improved) a much more than coincidental similarity between the 
TEL and the WBT. Nevertheless, are the two versions similar enough for 
valid cross-cultural coaparison or can they become such? 
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The ideal of identical similarity or 'identical twins' can probably 
never be attained, yet, which degree of similarity is practically 
attainable? Is the WBT perhaps already close to a realistic optimum? 
We have looked thus far without success for an answer to this question 
from other authors who have translated or adapted tests. We hope to 
find some clue when we are able to compare the distributions of 
answers from different groups of students in the USA, as soon as the 
American raw data are available to us. 

We make the following assumption; The siailarity of the reaction 
patterns of German and American students should correspond to the 
similarity of the reaction patterns of different American or different 
German-speaking students. The variance between the USA and German- 
speaking countries should come as close as possible to the variance 
within the USA or the FRG. 

Of course, the lack of similarity in the reaction patterns must not 
lead to the conclusion that the instruments are different. The 
reverse, however, can lead to the assumption that the instruments are 
similar if the reaction patterns are similar. The logic becomes 
clearer in the analyses which we were able to carry out thus far on 
the German material. 

Form A 

Different Groups of German Students 





TEL-A 


1 


2 


3 


4 


WBT A 


.62 










1 


.61 










2 


.57 


.82 








3 


.52 


.73 


.84 






4 


.55 


.78 


.89 


97 




5 


.57 


.82 


.98 


9o 


.92 



Form B 

Different Groups of German Students 
TEL-B 12 3 4 5 

WBT B .62 

1 .58 

2 .64 .89 

3 .65 .71 .85 

4 .64 .78 .89 .97 

5 .59 .82 .98 .90 .92 

Tab. 4-6: Correlations between and within different groups of 
students in the United States and in German-speaking 
countries. 



The correlation of r .62 between the TEL and the WBT seems pleasantly 
high, yet in view of the Intracorrelation of the German groups, it 
appears to us to be in need of improvement. 
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5- Correlation with external criteria 

The correlation of a test with external criteria is the most important 
criterion for validity. Up to the present time we have not been able 
to attempt any systematic study of this issue. Nevertheless, within 
the framework of the analyses carried out thus far, data have been 
gathered that give initial information about this issue. 

The data which we have so far gathered come from German, Austrian and 
Swiss students who were in groups with different kinds of economic 
courses. 

On the basis of their background, the averages of the German groups 
should have shown the following distinctions (Table 5-1): 

Expectations Results 
Rank Type of Students 

1. Universities 

2. Beruf soberschule 

3. Beruf sfachschule 
4* Beruf sschule 
^ • Wirtschaf tsschule 
6 • Wirtschaf tsschule-H . 



yes 
yes 
yes 
yes 
yes 
yes 



Rank Type of Students 

1- University students in the first term 

with two economic courses in the high school yes 

2. with one economic course in the high school yes 

3- without any economic course in the school yes 



Rank Type of Students 

1. University students who reads economic news 

in the newspaper yes 

2. University students who don't read 

economic np^s in the newspaper yes 

Tab, 5-1: Theoretically expected and actual differences in 
performance in different groups of students. 

Just as the TEL discriminates, as expected, between certain groups of 
students, ao does the WBT correspond to the theoretical expectations 
to a degree which we never would have dared to hope for. 

We are now confident that, by modifying the WBT slightly, we will have 
an instrument that allows us to carry out valid cross-national 
comparisons no matter which aspect of economic literacy the 
instruments measure. 
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6. A final look at the absolute data 

Table 5-1 might be somewhat frustrating since it only gives 
information about the larger/smaller relationships. In order to reduce 
any possible frustration, we would like to conclude by filling in some 
tables with their respective means and adding the little data from the 
US which we are aware of. 

However, we must first mention that, for the work on the adaptation, 
we administered the test to: 

1.351 German, Austrian and Swiss university students who, above all, 
wanted to become economics teachers 

2. 439 German and Austrian high school students with different kinds 
of courses in economics 

The data are certainly not representative and the interpretation of 
the data requires knowledge of the school systems in Germany and 
Austria which in comparison to those in the USA are very different. 
The limited space does not allow a description of the different school 
systems here. We limit ourselves, therefore, to two findings: 

1. First of all, we present the findings on the university students in 
Table 6-1. 

Adj. Mean std Dev 

Austria (A) 32.96 5.6 

Germany (FRG) 34.01 5.3 

Switzerland (CH) 36.89 5.8 

Tab. 6-1: Comparison of university students who want to become 
economics teachers (FRG, CH, A) 

2. Secondly, we compare the means of American high school students 
with courses in economics with the highly tentative (not 
representative) results of near peer-age students from Germany and 
Austria also with courses in economics. 
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form A 

German students: 

1. Universities 

2. Berufsoberschule 

3. Berufsfachschule 

4. Berufsschule 

5. Wirtschaf tsschule 

6. Wirtschaf tsschule-H. 

Austrian students: 



US- Students 
by Course Type: 

Economics 

Consumer economics 
Social studies 

overall 



economics 
hours per 
week (av) 



in. • • % of 
school curricu- 
years lum 



mean st.de v. 













33.26 


5.2 


6 


X 


2= 


12 


15,2% 


25.92 


5.4 


7 


X 


2= 


14 


21,2% 


25 28 


t ft 


6 


X 


2= 


12 


40,0% 


22.58 


6.1 


3,6 


X 


3= 


11 


8,6% 


14.66 


3.9 


5,5 


X 


2= 


11 


11,5% 


14.57 


2.7 


4,4 


X 


5= 


22 


13,5% 


23.24 


5.7 












23,57 


8.4 












21.70 


7.9 












22,85 


8.7 












23.33 


8.4 



Form B 



German students: 



economics 
hours per 
week (av) 



in. . . % of 
school curricu- 
years lum 



mean st.de v. 



1. 


Universities 












35.22 


5.6 


2. 


Berufsoberschule 


6 


X 


2= 


12 


15,2% 


28.80 


5.3 


3. 


Berufsfachschule 


7 


X 


2= 


14 


21,2% 


25.14 


2.4 


4. 


Berufsschule 


6 


X 


2= 


12 


40,0% 


22.28 


5.9 


5. 


Wirtschaf tsschule 


3,6 


X 


3= 


11 


8,6% 


17.04 


4.2 


6. 


Wirtschaf tsschule-H. 


5,5 


X 


2= 


11 


11,5% 


14.93 


3.3 




Austrian students: 


4,4 


X 


5= 


22 


13,5% 


27.25 


3.8 



US- students 
by Course Type: 

Economics 

Consumer econcnics 
Social studies 



overall 
Tab. 6-2: 



25.55 8.9 

18.07 6.9 

22.14 7.6 

23.92 8.6 



Comparison of the mean between high school students in 
US, FRG and A - with economics 
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The data still indicate almost nothing. However, they stimulate 
hypotheses and lead one to ask, "How is it really?". We hope we will 
soon have a valid answer. 
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